Material de la Charla del Evento de Virtualizacion del 10 de setiembre del 2009 en FUNDATEC.
http://ecastrom.blogspot.com
http://comunidadwindows.org
ecastro@grupoasesor.net
Ing. Eduardo Castro Martinez, PhD
Microsoft SQL Server MVP
http://ecastrom.blogspot.com
http://mswindowscr.org
http://comunidadwindows.org
As part of the Google Summer of Code, we tried to add support for SeaBIOS in order to allow guest OSes to be booted directly from PV disk devices rather than from the emulated disk device. SeaBIOS is the BIOS implementation that upstream qemu uses. When the virtual machine is created, SeaBIOS upon initialization uses a generic Xenstore client to communicate with the back end and initialize the front-end block device that will connect to the back end. After the connection is established I/O requests are made via the BIOS int 0x13 interface, guest OSes use the int 0x13 without needing to be aware that PV drivers were used.
This presenation gives a quick history on Hyper-V and discusses the arhcitecture of the vurrent release. It then goes into detail on Hyper-V R2, i.e. the build included in Hyper-V Server 2008 R2 and Windows Server 2008 R2. It includes Live Migration, Cluster Shared Volumes, Virtual Machine Queue, SLAT, Core Parking and Native VHD.
Material de la Charla del Evento de Virtualizacion del 10 de setiembre del 2009 en FUNDATEC.
http://ecastrom.blogspot.com
http://comunidadwindows.org
ecastro@grupoasesor.net
Ing. Eduardo Castro Martinez, PhD
Microsoft SQL Server MVP
http://ecastrom.blogspot.com
http://mswindowscr.org
http://comunidadwindows.org
As part of the Google Summer of Code, we tried to add support for SeaBIOS in order to allow guest OSes to be booted directly from PV disk devices rather than from the emulated disk device. SeaBIOS is the BIOS implementation that upstream qemu uses. When the virtual machine is created, SeaBIOS upon initialization uses a generic Xenstore client to communicate with the back end and initialize the front-end block device that will connect to the back end. After the connection is established I/O requests are made via the BIOS int 0x13 interface, guest OSes use the int 0x13 without needing to be aware that PV drivers were used.
This presenation gives a quick history on Hyper-V and discusses the arhcitecture of the vurrent release. It then goes into detail on Hyper-V R2, i.e. the build included in Hyper-V Server 2008 R2 and Windows Server 2008 R2. It includes Live Migration, Cluster Shared Volumes, Virtual Machine Queue, SLAT, Core Parking and Native VHD.
As time goes on more OSes are getting Dom0 support, so there's a growing need to provide a platform independent set of tools from which to operate Xen. This talk will expose the different mechanisms used on NetBSD that diverge from the Linux approach, and how Xen is improving its userspace tools to provide a more platform independent support.
The talk also touches upon various features that BSD provides or plans to provide with Xen, thus presenting a coherent roadmap view of where we've come from, and what lies ahead.
What's in this talk:
Xen and BSD
Status updates from the world of BSD
Ecosystem/userbase
This talk explores what has gone in so far in the Linux kernel (version 3.0 and 3.1) and which Linux distributions are deliverinbg Xen again. The otalk explores outstanding challenges and the pieces that are missing and what we can do, and what we cannot do working with Linux.
Cluster ist die Basis für die Private Cloud. In dieser Session lernen Sie die Erweiterungen der Cluster Services kennen, wie Cluster Shared Volumes, Cluster-Aware Updating und VM-Cloning und -Monitoring. Nach der Session können Sie die neuen Möglichkeiten einschätzen und das Potenzial für Ihr Unternehmen bestimmen.
Improving the Performance of the qcow2 Format (KVM Forum 2017)Igalia
By Alberto García.
qcow2 is QEMU's native file format for storing disk images. One of its features is that it grows dynamically, so disk space is only allocated when the virtual machine needs to store data. This makes the format efficient in terms of space requirements, but has an impact on its I/O performance. This presentation will describe some of those performance problems and will discuss possible ways to address them. Some of them can be solved by simply adjusting configuration parameters, others require improving the qcow2 driver in QEMU, and others need extending the file format itself.
(c) KVM Forum 2017
October 25 - 27, 2017
Hilton Prague, Prague, Czech Republic
http://events.linuxfoundation.org/events/archive/2017/kvm-forum
Learn about z/VM 6.2 Live Guest Relocation with Linux Middleware. A z/VM single system image (SSI) cluster is a multi z/VM system environment in which the z/VM
systems can be managed as a single resource pool and guests can be moved from one system to
another while they are running. For more information on IBM System z, visit http://ibm.co/PNo9Cb.
Visit http://on.fb.me/LT4gdu to 'Like' the official Facebook page of IBM India Smarter Computing.
In a traditional Xen configuration domain 0 is used for a large number of different functions including running the toolstack(s), backends for network and disk I/O, running the QEMU device model instances, driving the physical devices in the system, handling guest console/framebuffer I/O and miscellaneous monitoring and management functions. Having all these functions in one domain produces a complex environment which is susceptible to shared fate on the failure of any one function, has complex interactions between functions (including resource contention) which makes it difficult to predict performance, and has limited flexibility (such as requiring the same kernel for all device drivers).
""Domain 0 disaggregation"" has been discussed for some time as a way to break out domain 0's functions into separate domains. Doing this enables each domain to be tailored to its function such as using a different kernel or operating system to drive different physical devices. Splitting functions into separate domains removes some of the unintentional interactions such as in-domain resource contention and reduces the system impact of the failure of a single function such as a device driver crash.
Although domain 0 disaggregation is not new it is seldom used in practise and much of its use is focussed on providing enhanced security. Citrix XenServer will be moving towards a disaggregated domain 0 in order to provide better security, scalability, performance, reliability, supportability and flexibility. This talk will describe XenServer's “Windsor” architecture and explain how it will provide the above benefits to customers and users. We will present an overview of the architecture and some early experimental measurements showing the benefits.
As time goes on more OSes are getting Dom0 support, so there's a growing need to provide a platform independent set of tools from which to operate Xen. This talk will expose the different mechanisms used on NetBSD that diverge from the Linux approach, and how Xen is improving its userspace tools to provide a more platform independent support.
The talk also touches upon various features that BSD provides or plans to provide with Xen, thus presenting a coherent roadmap view of where we've come from, and what lies ahead.
What's in this talk:
Xen and BSD
Status updates from the world of BSD
Ecosystem/userbase
This talk explores what has gone in so far in the Linux kernel (version 3.0 and 3.1) and which Linux distributions are deliverinbg Xen again. The otalk explores outstanding challenges and the pieces that are missing and what we can do, and what we cannot do working with Linux.
Cluster ist die Basis für die Private Cloud. In dieser Session lernen Sie die Erweiterungen der Cluster Services kennen, wie Cluster Shared Volumes, Cluster-Aware Updating und VM-Cloning und -Monitoring. Nach der Session können Sie die neuen Möglichkeiten einschätzen und das Potenzial für Ihr Unternehmen bestimmen.
Improving the Performance of the qcow2 Format (KVM Forum 2017)Igalia
By Alberto García.
qcow2 is QEMU's native file format for storing disk images. One of its features is that it grows dynamically, so disk space is only allocated when the virtual machine needs to store data. This makes the format efficient in terms of space requirements, but has an impact on its I/O performance. This presentation will describe some of those performance problems and will discuss possible ways to address them. Some of them can be solved by simply adjusting configuration parameters, others require improving the qcow2 driver in QEMU, and others need extending the file format itself.
(c) KVM Forum 2017
October 25 - 27, 2017
Hilton Prague, Prague, Czech Republic
http://events.linuxfoundation.org/events/archive/2017/kvm-forum
Learn about z/VM 6.2 Live Guest Relocation with Linux Middleware. A z/VM single system image (SSI) cluster is a multi z/VM system environment in which the z/VM
systems can be managed as a single resource pool and guests can be moved from one system to
another while they are running. For more information on IBM System z, visit http://ibm.co/PNo9Cb.
Visit http://on.fb.me/LT4gdu to 'Like' the official Facebook page of IBM India Smarter Computing.
In a traditional Xen configuration domain 0 is used for a large number of different functions including running the toolstack(s), backends for network and disk I/O, running the QEMU device model instances, driving the physical devices in the system, handling guest console/framebuffer I/O and miscellaneous monitoring and management functions. Having all these functions in one domain produces a complex environment which is susceptible to shared fate on the failure of any one function, has complex interactions between functions (including resource contention) which makes it difficult to predict performance, and has limited flexibility (such as requiring the same kernel for all device drivers).
""Domain 0 disaggregation"" has been discussed for some time as a way to break out domain 0's functions into separate domains. Doing this enables each domain to be tailored to its function such as using a different kernel or operating system to drive different physical devices. Splitting functions into separate domains removes some of the unintentional interactions such as in-domain resource contention and reduces the system impact of the failure of a single function such as a device driver crash.
Although domain 0 disaggregation is not new it is seldom used in practise and much of its use is focussed on providing enhanced security. Citrix XenServer will be moving towards a disaggregated domain 0 in order to provide better security, scalability, performance, reliability, supportability and flexibility. This talk will describe XenServer's “Windsor” architecture and explain how it will provide the above benefits to customers and users. We will present an overview of the architecture and some early experimental measurements showing the benefits.
Running your Java EE 6 applications in the CloudIndicThreads
Session Presented @IndicThreads Cloud Computing Conference, Pune, India ( http://u10.indicthreads.com )
------------
The Java EE 6 platform is an extreme makeover from the previous versions. It breaks the “one size fits all” approach with Profiles and improves on the Java EE 5 developer productivity features. It enables extensibility by embracing open source libraries and frameworks such that they are treated as first class citizens of the platform. NetBeans, Eclipse, and IntelliJ provide extensive tooling for Java EE 6.
But how can you leverage all of this on a cloud ?
GlassFish v3, the Reference Implementation of Java EE 6, can easily run on multiple cloud infrastructures. This talk will provide a brief introduction to Java EE 6 and GlassFish v3. The attendees will learn how to create a simple Java EE 6 sample application and deploy them on GlassFish v3 running locally. Then it will deploy that sample using Amazon, RightScale, Joyent, and Elastra cloud infrastructures. It will also show how servers are dynamically provisioned in some environments to meet the demand. The talk will also explain the advantages of each approach enabling you to choose the optimal strategy for your environment.
Takeaways from the session
The attendees will be able to learn how to deploy a Java EE 6 application in different cloud environments. They’ll also learn about the pros/cons of these infrastructures.
A quick intro to DevCloud the CloudStack sandbox, and how to use CloudMonkey to manage your cloud.
DevCloud is a virtualbox image that contains the CloudStack source code and that is setup to run the storage infrastructure needed by CloudStack plus the networking setup to build the guest network of the VMs. Tiny Linux instances can be started within the Devcloud VM making use of nested virtualization.
This is a perfect setup to discover cloudstack, give demos and test new codes. It is used to test new releases and verify basic functionality. You can run DevCloud on your laptop and then use the command line interface CloudMonkey to make API calls to your DevCloud instance.
This is the perfect complement to the talk on CloudMonkey and shows the basic functionality of a cloud. Instance creation, snapshots, networking, network offering and AWS EC2 compatibility.
Presentation by Michael Van Horenbeeck: http://twitter.com/mvanhorenbeeck. Video recording available here: http://technet.microsoft.com/en-us/video/windows-server-2012-improvements-in-failover-clustering.
My talk from BACD http://buildacloud.org workshop in Ghent, Belgium
All videos can be viewed at: http://www.youtube.com/playlist?list=PLb899uhkHRoZZefRW5XmCb8QBcRO7o74E
This is an introductory talk for the workshop, it introduces CloudStack and the community at the Apache Software Foundation, it presents the basic layers of the Cloud IaaS, PaaS, and SaaS and shows how the CloudStack ecosystem addresses all layers. It presents the basic features of cloudstack, networking with a focus on SDN (Software Defined Networking) , storage with a focus on large scale object store (Ceph), a use case with Spotify, a PaaS with Karafe and fuse Fabric, the API using deltacloud which provides the CIMI standard interface and an application integration using the CloudStack API with Activeeon.
This is the perfect complement to the videos on youtube and serves as a introduction to CloudStack.
Turning OpenStack Swift into a VM storage platformwim_provoost
OpenStack Swift is the Object Storage project within OpenStack. Alas, due to technical hurdles (eventual consistency, blocks <> objects, …) it is impossible to run Virtual Machines directly on Swift. You need a layer in between Swift and the hypervisor which can overcome these hurdles. This is where Open vStorage comes in.
Open vStorage is an open-source VM storage router. It is a software layer (called the VM Storage Router) in between Virtual Machines and storage backends. It allows to abstract the backend from the Virtual Machine and creates a uniform, single namespace across multiple hosts. These VM Storage Routers (VSRs) operate like a grid leveraging local flash memory or SSDs and any storage back-end (S3 compatible object store, (distributed) filesystem, NAS) to provide an extremely high performance and reliable storage system.
One of the supported Object Stores is OpenStack Swift. Open vStorage is the only solution to turn OpenStack Swift into block storage for Virtual Machines. Through a Cinder Plugin it allows to create and manage volumes directly on top of OpenStack Swift. Combining Open vStorage with OpenStack Swift allows to create a scale-out, performing, VM-centric storage platform which neatly integrates with OpenStack.
Static partitioning is used to split an embedded system into multiple domains, each of them having access only to a portion of the hardware on the SoC. It is key to enable mixed-criticality scenarios, where a critical application, often based on a small RTOS, runs alongside a larger non-critical app, typically based on Linux. The two domains cannot interfere with each other.
This talk will explain how to use Xen for static partitioning. It will introduce dom0-less, a new Xen feature written for the purpose. Dom0-less allows multiple VMs to start at boot time directly from the Xen hypervisor, decreasing boot times drastically. It makes it very easy to partition the system without virtualization overhead. Dom0 becomes unnecessary.
This presentation will go into details on how to setup a Xen dom0-less system. It will show configuration examples and explain device assignment. The talk will discuss its implications for latency-sensitive and safety-critical environments.
XPDDS19: How TrenchBoot is Enabling Measured Launch for Open-Source Platform ...The Linux Foundation
TrenchBoot is a cross-community OSS integration project for hardware-rooted, late launch integrity of open and proprietary systems. It provides a general purpose, open-source DRTM kernel for measured system launch and attestation of device integrity to trust-centric access infrastructure. TrenchBoot closes the UEFI Measurement Gap and reduces the need to trust system firmware. This talk will introduce TrenchBoot architecture and a recent collaboration with Oracle to launch the Linux kernel directly with Intel TXT or AMD SVM Secure Launch. It will propose mechanisms for integrating the Xen hypervisor into a TrenchBoot system launch. DRTM-enabled capabilities for client, server and embedded platforms will be presented for consideration by the Xen community.
XPDDS19 Keynote: Xen in Automotive - Artem Mygaiev, Director, Technology Solu...The Linux Foundation
Artem will briefly cover what has been done since the first talk on Xen in Automotive domain back in 2013, what is going on now and what is still missing for broad adaptation of Xen in vehicles. The following topics will be covered:
Embedded/automotive features of Xen
Collaboration with AGL and GENIVI organizations for standardization
Efforts on Functional Safety compliance
Artem will also go over typical automotive use scenarios for Xen which may not be the same as generic computing use of hypervisor.
XPDDS19 Keynote: Xen Project Weather Report 2019 - Lars Kurth, Director of Op...The Linux Foundation
In this keynote talk, we will give an overview of the state of the Xen Project, trends that impact the project, see whether challenges that surfaced last year have been addressed and how we did it, and highlight new challenges and solutions for the coming year.
In recent years unikernels have shown immense performance potential (e.g., boot times of only a few ms, image sizes of only hundreds of KBs).The fundamental drawback of unikernels is that they require that applications be manually ported to the underlying minimalistic OS, needing both expert work and often considerable amount of time.
The Unikraft project provides a unikernel code base and build system that significantly simplifies the building of unikernels. In addition to support for a number CPU architectures, languages and frameworks, Unikraft provides debugging and tracing features that are generally sorely missing from unikernel projects. In this talk we will talk about these features, show a set of preliminary performance numbers, and provide a roadmap for the project's future.
XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software E...The Linux Foundation
The idea of making Xen secret-free has been floating since Spectre and Meltdown came into light. In this talk we will discuss what is being done and what needs to be done next.
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, XilinxThe Linux Foundation
This talk will introduce Dom0-less: a new way of using Xen to build mixed-criticality solutions. Dom0-less is a Xen feature that adds a novel approach to static partitioning based on virtualization. It allows multiple domains to start at boot time directly from the Xen hypervisor, decreasing boot times dramatically. Xen userspace tools, such as xl and libvirt, become optional.
Dom0-less extends the existing device tree based Xen boot protocol to cover information required by additional domains. Binaries, such as kernels and ramdisks, are loaded by the bootloader (u-boot) and advertised to Xen via new device tree bindings.
The audience will learn how to use Dom0-less to partition the system. Uboot and device tree configuration details will be explained to enable the audience to get the most out of this feature. The talk will include a status update and details on future plans.
XPDDS19 Keynote: Patch Review for Non-maintainers - George Dunlap, Citrix Sys...The Linux Foundation
As the number of contributions grow, reviewer bandwidth becomes a bottleneck; and maintainers are always asking for more help. However, ultimately maintainers must at least Ack every patch that goes in; so if you're not a maintainer, how can you contribute? Why should anyone care about your opinion?
This talk will try to lay out some advice and guidelines for non-maintainers, for how they can do code review in a way which will effectively reduce the load on maintainers when they do come to review a patch.
This talk is a follow-up to our Summit 2017 presentation in which we covered our plans for Intel VMFUNC and #VE, as well as related use-cases. This year, we will provide a report on what we have accomplished in Xen 4.12, and what remains to be addressed. We will also give a brief status update of VMI on AMD hardware. The session will end with some real-world numbers of the Hypervisor Introspection solution running on Citrix Hypervisor 8.0 with #VE enabled.
OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...The Linux Foundation
Safety certification is one of the essential requirements for software to be used in highly regulated industries. Besides technical and compliance issues (such as ISO 26262 vs IEC 611508) transitioning an existing project to become more easily safety certifiable requires significant changes to development practices within an open source project.
In this session, we will lay out some challenges of making safety certification achievable in open source and the Xen Project. We will outline the process the Xen Project has followed thus far and highlight lessons learned along the way. The talk will primarily focus on necessary process, tooling changes and community challenges that can prevent progress. We will be offering an in-depth review of how Xen Project is approaching this challenging goal and try to derive lessons for other projects and contributors.
OSSJP/ALS19: The Road to Safety Certification: How the Xen Project is Making...The Linux Foundation
Safety certification is one of the essential requirements for software to be used in highly regulated industries. The Xen Project, a secure and stable hypervisor that is used in many different markets, has been exploring the feasibility of building safety certified products on top of Xen for a year, looking at key aspects of its code base and development practices.
In this session, we will lay out the motivation and challenges of making safety certification achievable in open source and the Xen Project. We will outline the process the project has followed thus far and highlight lessons learned along the way. The talk will cover technical enablers, necessary process and tooling changes and community challenges offering an in-depth review of how Xen Project is approaching this exciting and and challenging goal.
XPDDS19: Speculative Sidechannels and Mitigations - Andrew Cooper, CitrixThe Linux Foundation
2018 saw fundamental shifts in security boundaries which were previously taken for granted. A lot of work has been done in the past 2 years, and largely in secret under embargo, but there is plenty more work to be done to strengthen the existing mitigations and to try to recover some performance without reopening security holes.
This talk will look at speculative execution sidechannels, the work which has already been done to mitigate the security holes, and future work which hopes to bring some improvements.
XPDDS19: Keeping Coherency on Arm: Reborn - Julien Grall, Arm ltdThe Linux Foundation
The Arm architecture provides a set of guidelines that any software should abide by when accessing the memory with MMU off and update page-tables. Failing to do so may result in getting TLB conflicts or breaking coherency.
In a previous talk ("Keeping coherency on Arm"), we focused on updating safely the stage-2 (aka P2M) page-tables. This talk will focus on the boot code and Xen memory management.
During this session, we will introduce some of the guidelines and when they should be used. We will also discuss how Xen boot sequence needs to be reworked to avoid breaking the guidelines.
XPDDS19: QEMU PV Backend 'qdevification'... What Does it Mean? - Paul Durrant...The Linux Foundation
For many years the QEMU codebase has contained PV backends for Xen guests, giving them paravirtual access to storage, network, keyboard, mouse, etc. however these backends have not been configurable as QEMU devices as their implementation did not fully adhere to the QEMU Object Model (QOM).
Particularly the PV storage backend not using proper QOM devices, or qdevs, meant that the QEMU block layer needed to maintain legacy code that was cluttering up the source. This was causing push-back from the maintainers who did not want to accept any patches relating to that Xen backend until it was 'qdevified'.
In this talk, I'll explain the modifications I made to QEMU to achieve 'qdevification' of the PV storage backend, how compatibility with the libxl toolstack was maintained, and what the next steps in both QEMU and libxl development should be.
XPDDS19: Status of PCI Emulation in Xen - Roger Pau Monné, Citrix Systems R&DThe Linux Foundation
PCI is a local computer bus for attaching hardware devices in a computer, and is the main peripheral bus on modern x86 systems. As such, having a proper way to emulate it is crucial for Xen to be able to expose both fully emulated devices or passthrough devices to guests.
This talk will focus on the current status of PCI emulation in Xen, how and where it is used, what are its main limitations and future plans to improve it in order to be more robust and modular.
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM SystemsThe Linux Foundation
Volodymyr will speak about TEE mediators. This is a new feature in Xen which allows multiple virtual machines to interact with Trusted Execution Environment available on platform. He developed mediator for one of TEEs, namely OP-TEE.
He will give background information on why TEE is needed at all and share some implementation details.
XPDDS19: Bringing Xen to the Masses: The Story of Building a Community-driven...The Linux Foundation
Xen is a very powerful hypervisor with a talented and diverse developers community. Despite the fact it's almost everywhere (from the Cloud to the embedded world), it can be difficult to set up and manage as a system administrator. General purpose distros have Xen packages, but that's just a start in your Xen journey: you need some tooling and knowledge to have a working and scalable platform.
XCP-ng was built to overcome those issues: by bringing Xen to the masses with a fully turnkey distro with Xen as its core. It's the logical sequel to the XCP project, with a community focus from the start. We'll see how it happened, what we did, and what's next. Finally, we'll see the impact of XCP-ng on the Xen Project.
XPDDS19: Will Robots Automate Your Job Away? Streamlining Xen Project Contrib...The Linux Foundation
Doug has long advocated for more CI/CD (Continuous Integration / Continuous Delivery) processes to be adopted by the Xen Project from the use of Travis CI and now GitLab CI. This talk aims to propose ideas for building upon the existing process and transforming the development process to provide users a higher quality with each release by the Xen Project.
XPDDS19: Client Virtualization Toolstack in Go - Nick Rosbrook & Brendan Kerr...The Linux Foundation
High level toolstacks for server and cloud virtualization are very mature with large communities using and supporting them. Client virtualization is a much more niche community with unique requirements when compared to those found in the server space. In this talk, we’ll introduce a client virtualization toolstack for Xen (redctl) that we are using in Redfield, a new open-source client virtualization distribution that builds upon the work done by the greater virtualization and Linux communities. We will present a case for maturing libxl’s Go bindings and discuss what advantages Go has to offer for high level toolstacks, including in the server space.
Today Xen is scheduling guest virtual cpus on all available physical cpus independently from each other. Recent security issues on modern processors (e.g. L1TF) require to turn off hyperthreading for best security in order to avoid leaking information from one hyperthread to the other. One way to avoid having to turn off hyperthreading is to only ever schedule virtual cpus of the same guest on one physical core at the same time. This is called core scheduling.
This presentation shows results from the effort to implement core scheduling in the Xen hypervisor. The basic modifications in Xen are presented and performance numbers with core scheduling active are shown.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
1. VastSky
Cluster storage system for XCP
Apr. 28th, 2010
VA Linux Systems Japan K.K.
Hirokazu Takahashi
Tomoaki Sato
Takashi Yamamoto
Xen Summit AMD 2010
2. What is VastSky all about?
VastSky is a cluster storage system
made up of a lot of servers and
disks, from which VastSky Manager
creates logical volumes for VMs
VMs can directly run on VastSky,
which XCP can control
VastSky is scalable, high availabile
and has a good performance
Xen Summit AMD 2010
3. XCP
ate
XML
r e q PC
cr e
ues
-R
t
VM VM VM VM VM VM
server agent agent control VastSky
Manager
logical volumes
l
tro
ate
n
co
cre
agent agent
server agent storage pool
Xen Summit AMD 2010
6. The way of making a logical volume
logical volume storage pool (physical disks)
server
server
Xen Summit AMD 2010
7. Good performance
All I/O operations will be done in
the linux kernel without any
VastSky Manager interactions.
I/O loads of logical volumes, which
can be extremely unbalanced, will
be equalized between the physical
disks.
I/O requests to rebuild mirrored
devices are also distributed across
a lot of physical disks.
Xen Summit AMD 2010
8. Load balancing of read/write requests
logical volumes storage pool (physical disks)
read/write
read/write
read/write
read/write
Xen Summit AMD 2010
9. Load balancing of read/write requests
logical volumes storage pool (physical disks)
read/write
read/write
read/write
read/write
Xen Summit AMD 2010
10. Mirrored disk recovery
Each mirrored disk doesn’t have its own
spare disk.
When VastSky detects one of the physical
disk chunks of the mirrored disk has
caused an error, VastSky Manager
allocates a new chunk form the storage
pool and assigned it to the mirrored disk
as a spare.
The manager schedules when it should be
assinged, so two or more re‐sync operations
won’t work on the same physical disk.
The mirrored disk starts re‐synchronizing
the disk chunks right after the spare is
assigned. Xen Summit AMD 2010
11. Mirrored disk recovery
mirrored disk (a part of a logical volume)
3.delete the failure disk 3.assign the chunk as a spare
BROKEN physical disk chunks
1. detect the error 2. allocate a chunk
storage pool
Xen Summit AMD 2010
12. Load balancing when re-synchronizing the
mirrored devices
When a certain physical disk gets
broken, VastSky tries to rebuild the
mirrored disks related to the
physical disk simultaneously since
the disk chunks belong to different
mirrored disks.
No need to rebuild if the disk chunk is
unsed.
Xen Summit AMD 2010
13. Load balancing when re-synchronizing the
mirrored devices storage pool (physical disks)
logical volumes
spare
spare
crashed
spare
Xen Summit AMD 2010
15. How to setup
VastSky should be installed with VM
management software such as XCP to take care
about VM life‐cycle.
Networking redundancy should be implemented
outside VastSky, such as using a bonding
device.
Hardware health check should also work
outside VastSky and hopefully it can tell
VastSky which server or disk should be
removed.
The current implementation of VastSky
requires HA cluster software to detect its
manager down to be restarted.
Xen Summit AMD 2010
16. API
VastSky supports XML‐RPC interface
and CUI like:
Define a logical volume.
Attach the logical volume on a
specified server.
Detach the volume.
Notify which disk or server has gone.
Add a new server or a physical disk.
Delete the server or the physical
disk.
Xen Summit AMD 2010
17. ToDo (1)
XCP integration
Under development.
Improve scalability.
Network topology aware volume allocation.
When creating a new logical volume,
physical disk chunks should be allocated
from storage servers close to the server
that owns the logical volume.
Logical volume expansion feature.
Snapshot feature for Guest volumes.
Xen Summit AMD 2010
18. Ideas of how to implement volume
snapshot feature
Use dm‐snap. It is the easiest way
but works slow.
Implement a completely new
implementation like Parallax does
but it will take long time.
Use OCFS2, which has rich features
but it will be a bit heavy.
Xen Summit AMD 2010
19. An idea of using OCFS2
If you place only one VM's volume
placed in an OCFS2 on a logical
volume on a head‐server, you can
obtain:
Better snapshot mechanism using an
ocfs2's new feature reflink.
Thin provisioning.
The volume can still be moved to
another server.
Xen Summit AMD 2010
20. Place only one guest's volume in
an ocfs2 filesystem
Server A Server B
VM1 VM2 migrate VM2 VM3
OCFS2 OCFS2
OCFS2
VM1's VM2's VM3's
volume volume volume
snap snap
migrate
snap
snap snap
snap
logical volume
logical volume
logical volume
can be extended
create
cre
create
ate
physical storage pool
Xen Summit AMD 2010
21. ToDo (2)
Shared storage for VMs, which some type
of active/active clustering software
requires. The point is the way of
rebuilding the mirrored devices.
The way to determine which server should
take the job to rebuild the mirrored device.
Make the rebuilding job and write access to
the device exclusively.
Fast VM deploying and cloning. This can
be done with the combination of “shared
storage” and “snapshot” features.
Xen Summit AMD 2010
22. Fast VM deployment
VM VM
logical volume logical volume
snapshot snapshot
cow cow
ready only ready only
e.g. CentOS installed
storage pool
Xen Summit AMD 2010
23. ToDo (3)
Make one server be able to manage both
VMs and a lot of physical disk.
Do you really want this feature?
Improve the disk chunk allocation
algorithm.
Make it disk performance aware.
Graceful server termination.
The copies of the chunks in the server
should be prepared before the termination.
Make VastSky Manager be able to run in a
VM.
Need some trick. The info to create the
volume of the VM for VastSky is stored in
this volume.
Xen Summit AMD 2010
24. Roadmap
First version release
XCP integration
Make it stable
Performance test
Write documents
The target date is this coming June.
Second version and after
The rest on the Todo list. What
should we do first?
Xen Summit AMD 2010
